Learning near-optimal policies with fitted policy iteration and a single sample path

In this paper we consider the problem of learning a near-optimal policy in continuous-space, expected total discounted-reward Markovian Decision Problems using approximate policy iteration. We consider batch learning where the training data consists of a single sample path of a fixed, known, persistently-exciting stationary stochastic policy. We derive PAC-style bounds on the difference of the performance of the policy returned by the algorithm and the optimal value function in both L∞ and weighted L-norms.